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INTRODUCTION 



This report is one of six Naval Postgraduate School technical reports 
documenting and describing a research project titled, f? Design of an Opera- 
tional Personnel Development and Evaluation System," sponsored by the Naval 
Material Command. The following is a listing of these six reports: 

1. NPS-55Gh73061 

DESIGN OF AN OPERATIONAL PERSONNEL 

DEVELOPMENT AND EVALUATION SYSTEM 

by: William H. Githens, Richard S. Elster, 

Gerald L. Musgrave, and John W. Creighton. 

2. NPS-55Ea7306l 

DESIGN OF OPERATIONAL CAREER LADDERS 
by: Richard S. Elster, Robert R. Read, 

William H. Githens, Gerald L. Musgrave, 
and John W. Creighton. 



3. NPS-55Gh73062 

DESIGN OF AN OPERATIONAL MANAGEMENT 
DEVELOPMENT MANUAL 

by: William H. Githens, Richard S. Elster, 

Gerald L. Musgrave, and John W. Creighton. 

4. NPS-55Gh73063 

DESIGN OF AN OPERATIONAL RATING MANUAL 
by: William H. Githens, Richard S. Elster, 

Gerald L. Musgrave, and John W. Creighton. 



5. 



NPS-55Mf 73061 

DESIGN OF AN OPERATIONAL MANAGEMENT BY 
OBJECTIVES MANUAL 

by: Gerald L. Musgrave, Richard S. Elster, 

John W. Creighton, and William H. Githens. 



6. NPS-55Re73061 

STATISTICAL ANALYSIS OF PERSONNEL DATA USING 
FACTOR SCORING, CLUSTER ANALYSIS, AND MULTI- 
DIMENSIONAL SCALING 

by: Robert R. Read, Richard S. Elster, 

Gerald L. Musgrave, John W. Creighton, 
and William H. Githens. 



An executive summary of the entire project follows, and any additional 
information about the project can be obtained from the Project’s Principal 
Investigator, Dr. Gerald L. Musgrave, Department of Operations Research and 
Administrative Sciences, Naval Postgraduate School, Monterey, California 
93940. 
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EXECUTIVE SUMMARY 
OF 

THE PERSONNEL DEVELOPMENT AND EVALUATION SYSTEM 



INTRODUCTION 

The purpose of this research project was to develop and implement a 
management system to more effectively utilize civilian professionals. 

Two n test bed” activities were selected — Naval Supply Center and Naval 
Regional Finance Center, both in San Diego, California. The project has 
four parts: 

1. Establishing a Group Appraisal System. 

2. Developing a Goal Setting System. 

3. Constructing Performance Rating Scales. 

4. Developing Career Ladders. 



GROUP APPRAISAL 

The management development program involved civilian professionals 
at the two commands in group performance appraisal sessions. A professional’s 
work performance was usually appraised by his supervisor and by the super- 
visor’s superior. This group would meet with a member of the research team. 
The appraisals conducted by these groups were focused on, and limited to, 
intra -appraisee considerations. That is, the appraisal committee considered 
the individual in terms of his greatest strengths and his least strong 
work performances, but did not compare the appraisee with other individuals. 
Recommendations for the appraisee, for the appraisee’s supervisor, and for 
the organization were then made so that this appraisee (a ’’human asset”) 
could grow in worth to himself and to the organization. 

A summary of the appraisal committee’s thinking was then written by 
the research team member who had attended the committee’s meeting and 
given to the appraisee’s supervisor for his review. The supervisor then 
discussed the appraisal with the appraisee, stating that this is ”how others 
see and interpret you,” and that ’’here are our thoughts on how you might 
further develop and utilize your talents.” 

The responses to the appraisal program were varied. A number of 
appraisees stated informally that they felt their appraisal session with 
their supervisor had been one of the most meaningful experiences they had 
while in the Civil Service. Many supervisors, however, experienced their 
first exposure to a face-to-face dialogue with one of their subordinates 
and found the feedback session to be somewhat traumatic. The development 
of supervisory skills in these feedback behaviors appears to be a crucial 
requirement if face-to-face dialogues between supervisors and subordinates 
are to become common and meaningful. 
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GOAL SETTING 



Another part of the project was to establish a framework to foster 
and facilitate a !, result oriented” management system. Our experience was 
that effective goals could be established and that while it took time to 
develop goals, the act of setting goals was beneficial to the organization. 

Goal setting was new to managers and they were resistant to formaliz- 
ing goals. Some of the resistance seemed to be attributable to unfamiliar- 
ity with the concept of producing results, as compared to being engaged in 
activities. Another resistive force seemed to be the fear that goal setting 
would be used for punitive managerial actions. 

We believe that after more experience is gained in goal setting and 
when employees’ fears of consequential management action are found to be 
unwarranted, a greater acceptance of the program will result. 

Our research at the Naval Postgraduate School and the San Diego 
Centers leads to the development of a new Goals and Controls System. This 
system includes a Work Performance Folder and a Goal Setting Manual that 
is to be used in conjunction with the folder. The system can be used to 
formulate goals, monitor and control performance, and to appraise work 
performance at the end of the year. 



PERFORMANCE RATING SCALES 

Section IV of this report presents the rating scales which were 
developed for professional occupations in Supply and Finance. 



ANCILLARY STUDIES 

The project report includes a number of sections which are indirectly 
related to the central issues of performance appraisal, goal setting, scale 
construction and career ladders. These related sections include analyses of 
questionnaires administered to individuals at the Centers, bibliographic 
resource materials, and a number of related ancillary studies. These 
studies are related to human asset accounting, goal setting, auditing, and 
statistical analyses of organizational climate and attitudinal data from 
the Centers. 
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BACKGROUND 



During Fiscal Year 1972, the Navy Material Command financed investiga- 
tions by Naval Postgraduate School (NPS) faculty as part of their explora- 
tory research directed at developing methods and means for improving organi- 
zational effectiveness. In the course of various dialogues concerning NAVMAT 
operations, topics related to the age and replacement of professional civilian 
personnel were discussed. These discussions then turned to the issues of 
performance evaluation and management by objectives. The Office of Civilian 
Manpower Management (OCMM) became interested in these problems, and the NPS 
was requested by NAVMAT and OCMM personnel to submit a proposal for implement- 
ing some relevant managerial programs during FY 73. NPS responded with the 
proposal included as Appendix 1 in NPS-55Gh73061 . 

The proposal involved the following main objectives: 

1. Developing for each civilian professional specific ways in which 
he can improve his knowledge, skills, attitudes, or behaviors to make him a 
more valuable human asset for the Navy. 

2. Develop for each civilian professional a list of specific ways in 
which management can better utilize his talent. 

3. Advise each civilian professional of what his boss wants him to 
accomplish during the coming year, and the evidence that will be used to 
judge such accomplishment. 

4. Generate for each professional position the best performance 
rating scales allowed by current technology. 

5. Generate "career ladders" for civilian professional jobs that 
relate field jobs to jobs in Washington, D.C. These "ladders" were to be 
based on the similarities and differences between and among jobs. 

The on-site locations for this "demonstration" project were the Naval 
Supply Center, San Diego, and the Navy Regional Finance Center, San Diego. 

The main administrative offices for both organizations are located in the 
same building and both organizations are served by the same personnel depart- 
ment. These two organizations were chosen because: (1) they are located 

in the same building, (2) this choice would allow one of the principal 
investigators to be on-site full-time, (3) they were within reasonable com- 
muting distance from the Naval Postgraduate School in Monterey, and (4) 
both were considered by NAVMAT and NAVCOMPT personnel to be relatively 
healthy and efficient organizations. 

A combination of "Management by Objectives" and "Group Appraisal" was 
used in accomplishing the first three of the five above objectives. Working 
from the higher toward the lower positions in the organizational hierarchy. 
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each supervisor called a committee meeting with his supervisor and several 
other employees who would have been in a position to observe the work 
performance of the appraisee. Following a brief discussion of the "strongest" 
and "least strong" aspects (intra-individual) of the appraisee's performance, 
the committee developed a list of recommendations in keeping with the first 
two of the aforementioned objectives. (Each of these discussions focused 
only on intra-individual differences.) Following this group meeting, the 
supervisor conducted a counseling session with the appraisee during which 
the opinions and recommendations of the committee were discussed. With this 
as a background, the supervisor and appraisee then worked out a list of 
specific goals for personal development to be accomplished during the coming 
year. In addition, based on the requirements and expectations of work 
accomplishment for the coming year as worked out by the supervisor and his 
boss, the supervisor and the appraisee (subordinate) worked out a list of 
goals for organizational accomplishment (objective //3) applying to the 
appraisee. Thirty of the 85 professional employees at NSC and all 25 of 
the professional employees at NRFC were covered by this program. Part II 
of this report deals with the developmental activities involved in objectives 
1 and 2, while Part III of this report is concerned with the MBO portion 
(objective #3) of the project. 

Generation of the best performance rating scales for each professional 
job (objective #4) involved the following scale construction steps: 

1. A group of employees (3 to 6) familiar with the job listed the 
most relevant aspects of performance for the specific job. 

2. The group then generated "specific" behavioral examples they had 
observed that demonstrated high and low performance on each performance 
aspect . 



3. At a later time, these behavioral incidents were presented to the 
individuals in the group, who assigned them to the rating scale (aspect) 
and rating scale level (low to high on a 5-point scale) that they thought 
appropriate. 

4. Incidents that were not by consensus assigned to the same location 
(both rating scale and level) were eliminated. 

This procedure yielded rating scales that are relevant to the job being 
rated and that are "anchored" by specific behavioral incidents representing 
on the scales the various levels of job performance. 

Rating scales were constructed for 6 of the 27 civilian professional 
jobs at NSC and for 3 of the 7 jobs at NRFC. General "supervisory" scales 
were constructed covering 11 of the 21 remaining jobs at NSC and all 4 of 
the remaining professional jobs at NRFC. Part IV of this report and Technical 
Report NPS55Gh73063 present the scale construction work conducted during 
the research project. 

In support of objective // 5, a task inventory asking employees to list 
the degree to which they were involved in various activities was administered 
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to 85 civilian professionals at NSC and 26 civilian professionals at NRFC, 

The same inventory was completed by civilian professionals in NAVSUP 
and NAVCOMPT in Washington, D.C. The data from the responses to this in- 
ventory formed the basis for the investigation of career paths, which was 
objective //5 of this project. The research done on career paths is described 
in Technical Report NPS55Ea73062 . 

Another technical report in this series, NPS55Rr73061, contains 
ancillary studies conducted during the term of this project. These studies 
included one using multidimensional scaling in examining how supervisors 
differentiate among their subordinates, and another effort which involved 
developing a comprehensive bibliography of the Management by Objectives 
literature . 
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MULTIDIMENSIONAL SCALING 
OF PERSONNEL PERCEPTIONS 



One of the objectives of the NSC/NRFC project was to investigate 
means for the selection of personnel who might be educated or trained 
for advancement. One of the criteria for such a selection is the abi- 
lity of the person to get along with other people. This quality in- 
volves the respect of others and the ability of the person to fulfill 
the requirements that others perceive to be important in a job. 

Such a quality is difficult to measure, for it depends upon the 
opinions of people who can be expected to be reluctant to state whether 
or not a person’s characteristics are good or bad and why they are good 
or bad. Since a direct measurement approach is difficult, the study 
team decided to use a system which would give such a measure without 
asking any direct questions of personnel in the organization. This 
technique is called multidimensional scaling. In this system, the 
subjects need only to identify how similar or dissimilar they perceive 
their colleagues as being to one another. These comparisons are usu- 
ally made by having the respondent consider one pair of subjects at a 
time. 



A simple example will help to illustrate the technique. Consider 
five personnel designated A, B, C, D, and E, and suppose each is asked 
to state, on a psychological scale of 0 to 9, how far apart from one 
another they perceive the employees in each pair as being. Thus, one 
person might perceive A and B to be very similar and scale that pair 
as ’’one”, whereas A and C may be perceived as being very dissimilar 
and hence scaled at ’’eight”, etc. These scalings can be averaged over 
the individuals who respond in order to produce a half matrix such as 
the following: 



A 0 

B 2.5 0 

C 5.0 3.0 0 

D 5.5 4.5 2.5 0 

E 3.5 3.5 3.5 2.5 

A B C D 



0 

E 



Note that no individual is asked to state which member of a pair is 
’’better”, nor is he identified with his response (anonymous responses). 
The above half matrix of averages, therefore, represents an average of 
the group’s perception. 

Data of this type resemble a mileage table that often appears on 
road maps. In fact, one can view the data as such and try to recon- 
struct positions of the cities on the map. Thus, using the fictitious 
data from above, points A, B, and C must lie on a circle of radius 3.5 
from E. Points A and C must be 5 units apart while B is about 2.5 units 
from A, and all three of these are on the originally constructed circle. 
Proceeding in this fashion leads to the following comparative locations 
for the points: 
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Hence, the goal of achieving a spatial representation (locating the cities) 
of the five personnel has been achieved. In this example, the ten entries 
in the data set were contrived so that the two-dimensional construction 
process could be completed without any substantial inconsistencies. Un- 
fortunately, such tidy solutions do not too often happen with this kind of 
data and more than two dimensions may be needed to represent the points 
geometrically. 

Having achieved the above representation, how does one interpret it? 
First, it would show that there are two main dimensions underlying the 
group’s perception of their colleagues. What remains is the interpreta- 
tion of these two dimensions. Suppose it is learned that the group’s 
supervisor considers A to be a person of high productivity, C and D are 
of low productivity, while B and E are moderate producers. Then, one 
would feel justified in constructing a scale and labeling it "p” for 
productivity as has been done in the previous diagram. Suppose further 
that it is learned that C is a very thorough individual, E is sloppy and 
careless, while A, D, and B are rated as being moderately thorough. Then 
one would feel justified in constructing and labeling the ”t”-scale (for 
thoroughness) as was done on the diagram. Thus, the goal of this multi- 
dimensional scaling technique is to discover and represent the dimensions 
of perceptions. 

In this example, two things have been exposed that may not have been 
otherwise learned. First, the group perceives productivity and thorough- 
ness as the important dimensions separating the personnel in that group. 
Second, the perceived positions of individuals on these dimensions have 
been learned. 

It was decided to try this multidimensional scaling technique with 
some San Diego Supply personnel in order to demonstrate its possible 
usefulness. Discussion with Supply personnel in San Diego led to the 
selection of the following five groups for this pilot study: 
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Code 3051: Screen I.D. and Special Projects 

Code 1011: Inventory Management 

Code 5311: NSF/Reconciliation 

Code 5232: Allotment Sec, III 

Code 5332: Payroll and RAM 



(7 people) 
(8 people) 
(7 people) 
(7 people) 
(8 people) 



There was prepared for each group (code) a lower trangular matrix 
with the names of the employees heading the rows and columns, analogous 
to a mileage table on a map in which the rows and columns are headed by 
cities and entries in the matrix represent distances between the cities. 
In fact, this analogy was used to instruct the respondents. Instead of 
distances between cities they were requested to enter distances between 
pairs of fellow workers. These distances were to be on a psychological 
scale of 0 to 9 and in terms of "value to the organization". Several 
asked for a more explicit breakdown as to what this meant, and it was 
explained that such determinations were to be made by the respondents. 
Indeed, the purpose of this exercise is to determine the number and 
character of the important dimensions perceived by the group in common. 

This latter point seemed difficult to get across. Many respondents, 
upon learning that zero meant that no difference in the two individuals f 
values to the organization could be perceived by them, proceeded to en- 
ter all zeros. Others were concerned about how to designate which mem- 
ber of the pair was better. The reply was that it was not necessary to 
so state, but only necessary to indicate how far apart they were. It 
appeared to the researches that not all of the respondents were comfort- 
able with this answer. Finally, it was necessary to instruct codes 5311, 
5323, and 5332 all at once and in a room where no one could sit down. 

This fact contributed to a lack of communication and cooperation. 

The data were collected, processed, and received with each code’s 
supervisor. The results of each review follow immediately. After 
these, a summary is presented. 



Code 5323 has seven women and a woman supervisor. One was absent on 
the day of the data collection, and the remaining six responded. A two- 
dimensional solution was found satisfactory and was readily interpreted 
by the supervisor. The representation of the solution and the dimen- 
sions are presented in Figure 1. 

The supervisor’s interpretation follows: 

E and F are people of high productivity, initiative, and seem to 
know how to handle their problems. 

A is an individual of low productivity, who gripes, and wants 
tangible rewards for doing something extra. 



CODE 5323 
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FIGURE 1 



CODE 5323 




D and B do very average work, have similar personalities and are 
somewhat indecisive. G is new and hence unknown to the group. C is 
a physically handicapped person who is doing the best she can. She is 
not very mobile (e.g. can’t leave her desk to reach the filing cabinet) 
and this is seen as influencing her productivity. 

Thus, the two dimensions appear to be Productivity and what we 
shall call Self Reliance. 

CODE 3051 



Code 5031 has seven men (A through G) and is supervised by a man. 

All seven supplied data, but some of the forms were incomplete. It was 
necessary to go to three dimensions to get a satisfactory solution. The 
diagram (Figure 2) exhibits a string of four people: F, G, B, and C, 
with a fifth, D, along the same line, but D is much lower in the vertical 
plane. This grouping is flanked by A on one side and E on the other. 

A discussion with the supervisor yielded the following interpreta- 
tions : 

F is a new man (he had been with the organization about a year and 
a half). He is quiet, plugs along, and not very knowledgeable about 
marine applications of the equipment. He came here as the result of 
a RIF (elsewhere) and would not, under ordinary circumstances, have 
been selected for his position because of the investment in training 
involved. 

C is the most knowledgeable man in the section. He is ambitious, 
wants a promotion, likes to do new and different things, and has a ten- 
dency to socialize (swap stories with others) . C and F are the only 
two in the section who are not close to eligibility for retirement. 

B and G are quite similar and close to the middle in terms of value 
to the organization. D is very steady. He doesn’t like new jobs or to 
make decisions. He is a loner (non-social) and "moans and groans". 

A and E are the top "doers" of the section. They both are go-get- 
ters who show lots of initiative and frequently draw special jobs. 

It was very surprising to find that A and E are perceived by the 
group as being so different, so the point was pursued further. A is 
rough cut and a leader ("Let’s get off our tails and get the job done") 
whereas E is more of a diplomat. Also, many of E’s work assignments 
take him out doors away from the gang. A has lunch with the "core" 

(C, D, B, and G) , whereas E eats alone. The supervisor suspected that 
this lack of group contact is the dimension that splits E from A in the 
perception of the group. 
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FIGURE 2 



It was difficult to label the third dimension of these data. The 
main theme appeared to be knowledgeability , while the A to E direction 
suggested a personality or perhaps leadership/popularity dimension. 

The withdrawn conservative is low in the vertical, whereas the gregari- 
ous types seem rather high. The fact that the representation cannot be 
more sharply described suggests that there may be perceptual ingredients 
present of which the supervisor is not aware. 

CODE 5332 



Code 5332 has eight people, two of which are males (A and H) , and a 
female supervisor. One person was absent on the day of the data collec- 
tion, and five of the remaining seven turned in data sheets containing 
all zeros. Thus, either they were unable to differentiate among their 
fellow section members, or they were unwilling to share any such percep- 
tions with the researcher. Also, there were many zeros on the two data 
sheets that did contain information. It was decided to complete the 
analysis even though the data are a very small sample. 

A two-dimensional solution was found to be satisfactory and the 
results were readily interpreted by the supervisor. 

Individuals F, G, and H are retirement clerks. The others are pay- 
roll clerks and the two subgroups have virtually no professional inter- 
action. The two subgroups are clearly separated in Figure 3. 

F, the head retirement clerk, has overall responsibility, is very 
exacting and does everything "by the book". H resents this, has sloppy 
work habits, and frequently is made to do things over again. G is in 
between, and just goes along. 

Turning to the other subgroup: B is slow, thorough, and very ac- 

curate in her own way, but it is not possible to get her to do things 
the way others do them. C and E are quite similar to one another, 
also, they are the oldest and are felt to resist change. A and D are 
both high on initiative and volunteer for extra work. D is a trainee 
while A (a male) has performed work similar to this for many years. 

Thus, the plot really has two parts, one for the retirement 
group — which is essentially one dimensional and represents sharp 
attention to detail. The other plot is for the payroll group which 
has two dimensions — one for experience, and one for a combination 
of what we shall call Speed and Conventionality. 

CODE 5311 



Code 5311 has seven people, two of which are male (A and F) , and 
a female supervisor. One was absent the day of our data collection, 
and three turned in data consisting entirely of zeros. Again, the 
analysis was conducted even though the sample was very small. 
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FIGURE 3 



CODE 5332 




FIGURE 4 



CODE 5311 




F is a most aggressive individual who will not rest until he solves 
a problem — he ’’digs". D was trained by F and is also a "research type". 
G is a very demanding and stubborn person who is low on initiative and by- 
passes the organizational structure when she has a complaint. A is a 
quiet, non-demanding, non-aggressive individual who works steadily and 
gets nervous if pushed. 

The supervisor would rate B and C about the same. C has been longer 
on her job and her output should be greater, so perhaps the group has 
picked up on that situation. E has an "OK" personality and also has not 
been on the job as long as C. 

Thus, the two dimensions in Figure 4 appear to be initiative and 
some sort of intenseness or energy expenditure dimension. The super- 
visor may wish to look farther in interpreting the separation of B and C. 

CODE 1011 



Code 1011 has seven women and one man, H. The supervisor is a woman. 
One was absent on the day of the data collection and only three of the 
remaining seven supplied discriminating information. Even so, three 
dimensions were necessary to obtain a satisfactory representation. 

Obviously, the man, H, is sharply separated from the rest. 

The supervisor felt that the separation of B and D was astounding. 

D is the youngest and probably one of the best. Some feel that D gets 
preferential treatment. B is very conscientious and does rather dif- 
ferent work. E, F, and G have been on the job a long time (about 25 
years or so) . E has the best overall knowledge, but has personality 
problems (does not get along well and can’t communicate). F is very 
outgoing. C is friendly, outgoing, and does a good job. A is with- 
drawn, keeps to herself, and does not have broad experience. H is 
also withdrawn. 

The supervisor felt that the separations exhibited (see Figure 5) 
are rather special. She sees clustering by personalities, car pools, 
and who lunches with whom. No attempt was made to interpret the dimen- 
sions found for the data from this code. 

SUMMARY 



Code 5323 had a good response rate and an easily interpreted solution. 
This suggests that there is good rapport between the supervisor and the 
people in the code. 

Codes 1011, 5311, and 5332 had a poor response rate. This may be at- 
tributed to poor instruction, poor understanding, or poor attitude. Even 
so, the results of 5311 and 5332 were interpretable, although the results 
were based on a few responses. The results from 1011 seem useful only in 
telling us that those who did respond may have views different from those 
of the supervisor. 
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CODE 1011 



FIGURE 5 
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Code 3051 provided a good response rate, but only two of the three 
dimensions were interpretable by the supervisor. Thus, he is alerted 
to the possibility that an ingredient of rapport may be missing between 
him and his people. Also, he was surprised to learn that his two best 
"doers" were split apart in the perceptions of the group. 
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USE OF FACTOR SCORING AND CLUSTER ANALYSIS 



WITH DATA FROM THE OCMM SURVEY 



In addition to the application of multidimensional scaling, cluster 
analysis and factor scoring techniques were applied to personnel data 
collected at the Naval Supply Center, San Diego. The Navy Office of 
Civilian Manpower Management (OCMM) developed the Civilian Personnel 
Management Survey as an instrument to aid in the self-evaluation of 
various Navy shore activities. The questionnaire focused on civilian 
personnel management programs. Each civilian in the command is asked to 
respond to sixty-five statements about the actual operation of the organ- 
ization. A list of questions is presented in Table 6. The response 
choices are to agree, disagree or be undecided about the statements. 

Two forms are used, ,, supervisors M and "employees , " and responses are 
made on separate optically scanned answer sheets. The results of the 
responses are centrally tabulated and returned to the activity. An 
example of one tabulation is presented in Table 2, 

To aid the activity, the sixty-five questions are categorized into 
eleven "program areas" which are used to identify a variety of personnel 
management programs. These eleven areas are Merit Promotion, Training, 
Labor-Management Relations, E.E.O. , Classification and Pay, Position 
Management, Job Information and Performance Evaluation, Communication, 
Supervision, Employee Services, and Morale. In addition to the raw 

scores, the command’s responses are compared to the Navy average. 

The scoring system operates as follows. For each question, a 
determination is made as to whether an "agree" or "disagree" response 
is supportive (indicative of satisfaction) with the present operation 
of the activity. Scores (percentage responses) on the so-designated 
answers are compared with Navy average responses on a question-by- 
question basis. For each program area, the total percentage differences 
over all questions is divided by the number of questions in that program 
area. From these data it is determined how the activity compares with 
the Navy average for each program area. An example of this summary is 
provided in Table 1. OCMM feels that there are no right or wrong answers. 

The results are to be used to identify those areas where there is a high 

degree of acceptance and support, and areas which may be in need of 
better communication, redirection, or other management action. 

Further insight into the details, philosophy, development, and 
progress of this survey can be obtained from the following four articles: 
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1. Masse, S., "Evaluation — A New Generation Concept," Civilian 
Manpower Management , Vol. Ill, No. 2, Summer 1969, pp. 22-24. 

2. Berne, E., "Evaluation — A New Generation Concept Progress 
Report I," Civilian Manpower Management , Vol. IV, No. 3, Spring 1970, 
pp. 27-30. 



3. Froscher, C.T., Capt. USN, "Evaluation— A New Generation 
Concept Progress Report II, M Civilian Manpower Management , Vol. IV., 
No. 2, Summer 1970, pp. 20-29. 

4. Masse, S., "The Questionnaire Survey Technique," Civilian 
Manpower Management , Vol. IX, No. 3, Fall 1971, pp. 6-11. 



It would seem reasonable to believe that the presentation of the 
OCMM survey data could be simplified to a substantial extent. That is, 
the manager who uses the results might be able to glean the information 
garnered by the survey without having to examine the responses to all 
the items in an area or in the survey. The techniques of cluster analy- 
sis are exploitable for this purpose. Indeed, Masse* has pointed out 
that "Numerically significant clusters of responses should clearly 
point up areas for further investigation." Also, when administrative 
action is taken in an area, the perception of the affected personnel 
could change not only there but in other areas as well. Thus, action 
should be directed toward all the personnel who have common perceptions, 
if possible. The purpose of this study is to illustrate just how such 
clusters of personnel (responses) can be identified. 



More specifically, the use of cluster analysis in a pilot study 
(1972 OCMM survey for NAVSUP, San Diego) has led to the identification 
of six clusters of personnel. Each cluster is scored on each program 
area (see Figure 1) and those clusters contributing to unusual perform- 
ances exhibited in Table 1 are identified. Such identification occurs 
only through the associated personnel profiles since the original re- 
sponses are anonymous. As a result of the pilot study, the following 
items have been brought out : 

1. The high performance in Labor Management Relations is due to 
the combined perceptions of clusters C2 and C5. This same pair causes 
above average ratings in Position Management. These two clusters have 
more than their share of twenty-year people (see Table 5). 



^ Journal of Navy Civilian Manpower Management , Vol. Ill, No. 2, 
Summer 1969. 
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2. The above average rating in Communication is due to the per- 
ceptions of clusters Cl, C4, and C6 . No military personnel are identi- 
fied with these three clusters, and Cl and C4 have a sharply dispropor- 
tionately high number of females. Curiously, C2 and C5 perceive com- 
munication to be relatively poor. 

3. The two pairs, C1-C4 and C2-C5, behave as two single clusters 
for most program areas. The most singular exception is their scores on 
Equal Employment Opportunity. There, Cl and C5 pair up (with C6) to 
drive the above average rating while C2 and C4 detract from it. This 
particular area is governed virtually entirely by the response to Item 
19 - "minority members perform as well as others". One might expect 
that these clusters would have a disproportionate number of minority 
personnel, but the numbers are not significant (see Table 5). There 

is a moderate tendency to associate higher education with C5. 

Deeper probing is possible and is illustrated later on. The tech- 
nique is presented first. The following paragraphs provide an overview 
of the data reduction steps that could be used to locate clusters. Af- 
ter that, these steps are applied to the CY 1972 responses with broad- 
based normative data, hence the illustrative results are limited to an 
internal (NAVSUP) comparison. 

The objective of the data analysis method is to group the respond- 
ents into clusters of individuals who gave similar responses to the 
questionnaire. Obviously, if we required all members of a given clus- 
ter to have exactly the same responses on all 65 items in the OCMM sur- 
vey, then one would have a very large number of clusters, and no parti- 
cular advantage would have been gained from the clustering. One desires 

a handful of clusters such that within a cluster the responses of the 
individuals in the cluster are very similar, and yet there are substan- 
tial differences among the responses of individuals in different clus- 
ters. Computational routines* are available for forming clusters, but 
generally these methods are not computationally feasible if the number 
of items is larger than 20. This problem can be circumvented by first 
grouping the 65 items into a set of "factors" so that the number of 

such factors is under 20. Each respondent must then be assigned a 

score on each factor. One can help compensate for information lost in 
this preliminary grouping by choosing a scoring system that best cap- 
tures the differences among the respondents 1 answers to the OCMM sur- 
vey. The technique chosen involves the items* being weighted in such 
a way that the variability among the responses to the survey is maxim- 
ized. 

The clustering procedure is applied to the correlations among the 
scores received by the respondents on the factors that were developed. 
Objective criteria are available for determining the number of clusters 
having substantial inter-cluster separation and minimal intra-cluster 



*McRae, D.J., Clustering Multivariate Observations , Ph.D. Thesis, 
Univ. of N. Carolina, 1973. 
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separations. The individuals in any one particular cluster are those 
who gave quite similar responses to the OCMM survey. Having developed 
the clusters, one then tries to describe the clusters in terms of what 
responses the people in them had in common. 

An Example : 

For illustrative purposes, the procedures outlined above were used 
to examine the CY 1972 responses of 106 supervisory personnel at NSC, 

San Diego, to the OCMM survey. The procedural details chosen were 
somewhat ad_ hoc , but quite reasonable. 

1. It was both convenient and expedient to use the eleven program 
areas (see Table 1) to serve as the factors. The items associated with 
each program area appear in Table 3. Such a choice represents the way 
that OCMM feels its survey is organized, although they make no claim 
that it results in sharp item groupings. Thus, further study of the 
groupings might be helpful. 

2. Scores for each factor were formed by using the item weights 
that appear in Table 2. The scoring method used chose the item weights 
so that the variances of the factor (program area) scores were maximized. 
In this way, the discrimination among the clusters should have been made 
large. The item responses were converted to numbers by assigning 0 for 
n yes", 1 for "?", and 2 for ,f no n . Missing responses were converted to 
"?" (or 1). Each numerical response was multiplied by the weight for 
that item, and program area scores were computed by adding these prod- 
ucts over all items in the area. 

3. A six cluster solution was chosen since it produced a noticeable 
break in the within group sum of squares, and Mahalanobis distance was 



used (see McRae, 
given by: 


op. cit 




The 


number of 


personnel in each cluster is 


Cluster 

Number 


1 


2 


3 


4 


5 


6 Total 


Number of 
People in 
Cluster 


12 


28 


10 


15 


35 


6 106 


A visual display 


of the 


mean 


value of 


each 


factor for each cluster is 



given in Figure 1. 

4. Normative data for this example are limited to the overall means 
for each factor. The scoring interpretations that appear in Table 4 pro- 
vide tentative interpretations of the program area scores. Based on 
Table 4, the clusters are interpreted in the following several paragraphs. 

It can be seen in Figure 1 that areas 7 and 11 (Job Information and 
Performance Evaluation, and Morale) provide minimal discrimination among 
the clusters. Note further that the two largest clusters, C2 and C5, 
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which account for 63 of the 106 supervisors, have similar centroids on all 
factors except area 4, EEO. In Table 3, it can be seen that this area re- 
lates highly to Item 19 ("Minority members perform as well as others") . 

It follows that cluster C5 consists largely of those supervisors who agree 
with the statement that minority members perform as well as others, and 
that cluster C2 contains those that disagree with it. On the basis of 
these data, one cannot state whether this difference is one of perception 
or of fact, but having called attention to the issue, one can decide if 
further investigation is needed. Also note that the people in both clus- 
ters C2 and C5 feel communication is poor. 

The smallest cluster, C6, is a sharp outlier, having extreme positions 
on areas 1, 2, 6, 8, and 9. The people in C6 seem dissatisfied with the 
merit promotion program and not too happy with training, have few or no 
complaints about minority workers, feel communication is good, but feel 
their authority is lacking, and their high score on area 9 suggests dis- 
satisfaction with their supervisory situation. 

Clusters Cl and C4 are very close to each other on areas 1, 2, 3, 5, 

6, 8, and 9, but separate on area 4 (EEO). Thus, the people in these 
clusters have some complaints about the merit promotion program, training, 
are somewhat anti-union, do not feel that employees leaving for higher pay 
is a problem, but feel they need more authority, and have some dissatisfac- 
tion with supervision. 

The remaining cluster, C3, has 10 members and seems to have an inter- 
mediate position on all areas, except possibly morale (area 11), an area in 
which their attitude appears a bit negative. It is curious that this group 
has a conspicuously neutral position on area 4, EEO. 

Having characterized the clusters, one is in a position to make in- 
depth studies of the extreme positions of clusters on areas. Thus, one 
might want to break down the item responses of C2 on all items in area 
4 (EEO) . The responses of C6 on area 1 (Merit Promotion) and area 8 
(Supervisory) seem to be extreme and worthy of special attention, and so 
on. 



5. To learn more about the clusters, one can summarize the demo- 
graphic (profile) and other data available concerning people in the clus- 
ters. Some salient frequency counts appear in Table 5. For instance, 
from Table 5 we see that Cl and C4 have sharply disproportionate numbers 
of females, while the other differences between the people in the two 
clusters appear to be rather minor. C2 and C5 have an excessive number 
of 20-year people. 

Cross-classification also can be made for purposes of typing or 
understanding the clusters. For example, cross-classifying education 
with work area for C5 yields the following table: 
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Scien 
& Eng 


Other 

Prof. 


Admin 


Technical 


Clerical 


Trade 


Total 


High School 


0 


1 


i 


2 


2 


6 


12 


College 


0 


3 


i 


2 


0 


0 


6 


Postgraduate 


0 


0 


3 


0 


0 


0 


3 

21 



Thus, all those that have postgraduate training are administrators, all 
supervisors in the clerical or trade groups have a high school education, 
etc. It is possible that such comparisons could help in understanding 
the differences among the responses given by people in different clusters. 

In summary, it is hoped that this example illustrates the kinds of 
analyses that are possible through cluster analysis. One can define 
clusters from the responses themselves, and can discover important dif- 
ferences in perceptions and attitudes — helping to identify the human 
and organizational variables associated with such differences. 



21 



Denote* Overall Mean 



- t N 10 

o o o po 



re 



• » * 

o o o 



* M 

1° ° 

I 7J 

o 



< 

liJ 

Od 

< 

2 E 

< 

a: 

o 

o 

a: 

o- 



o o o" 



4 - 



ayoos 



a o 



O O O I o 



o 



w n 
O O 



- oV v 



* ^ 
O o 



cf tT CJ 



« » N 

o o a 



o a 



O O 



• * ** n m 

i°. «p 



10 

O 



O * _ 

O o 



_ * • in 

o a o o 



N 

o 



o 

o I 0»0* o" 
• T 



<0 

o 



♦ — 10 » 

O O O I o o 






o 

“«>/ 






a>< 



V. 



V *x 



00 



\ 

*°3 

v% 

♦a. * » 






m 






'^x 



fO 



•to 









CM 



O 00U)^NO00(Ot N O CVJ ^-<o 

cm — — — — dodo 6 6 66 



22 



Figure 



TABLE 1 



QUESTIONNAIRE RESULTS 
NAVSUP San Diego, May 1972 
Comparisons with Navy Average 



E: Employees 

S: Supervisors 



Program Area 


Sharply 

.Above 

(10% or more) 


Above 
(5% to 9% 


About 
The Same 
(Under 5%) 


Below 
(5% to 9%) 


Sharply 

Below 

'10% or more) 


Merit Promotion 


E O 


EO 


E# 


EO 


eO 




SO 


SO 


sA 


SO 


sO 


Training 


EO 


EO 


E # 


EO 


EO 




so 


SO 


s A 


SO 


SO 


Labor-Mgt. Relations 


EO 


eQ 




EO 


EO 




s 


sO 


sO 


SO 


so 


EEO 


EO 


ecT~^~ 




EO 


EO 




so 




so 


SO 


so 


Classification & Pay 


EO 


EO 




EO 


EO 




SO 


SO 


s m 


SO 


so 


Position Management 


EO 


EO 


r .A 


EO 


EO 




sO 




SO 


sO 


so 


Job Info/Perfomance 


EO 


EO 




eO 


EO 


Evaluation 


SO 


sec*" 


so 


sO 


so 


Communication 


EO 


EO 




EO 


EO 




so 


s®cc^ 


' so 


SO 


SO 


* 

Supervision 


EO 


EO 




EO 


EO 




SO 


so. 


sm 


SO 


so 


Employee Services 


EO 


EO 


E 0 


eO 


EO 




so 


SO 


sp 


SO 


so 


Morale 


EO 


EO 




eO 


EO 




so 


s 0 — 


so 


SO 


so 
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TYPICAL PROFILE DATA S 



TABLE 3 



PROGRAM AREAS OF SUPERVISOR RESPONSE ITEMS AND CORRESPONDING SCORING WEIGHT 
Item 

No. Weight 

1. MERIT PROMOTION 

1) Satisfied with people referred .16 

11) Good applicants to choose from -.13 

18) Candidates in reasonable time .41 

25) Trained in supervisory appraisals .38 

2. TRAINING 

2) Job-related and pays off -.16 

6) Can get for employees in reasonable time .30 

32) Difficult to spare employees for (off-job) .36 

3. LABOR-MANAGEMENT RELATIONS 

9) Personnel Office assistance in dealing with unions .10 

35) Kept informed of provisions of agreements .38 

44) Feel free to treat union members /non members the same .37 

54) Satisfactory dealings with unions -.03 

4. EQUAL EMPLOYMENT OPPORTUNITY 

7) Supported by top management and supervisors -.06 

16) Men and women have same job opportunities -.01 

19) Minority members perform as well as others .59 

31) Would mind working for a minority supervisor .05 

40) Minority members treated fairly -.08 

5. CLASSIFICATION AND PAY 

13) Known procedure when PD is out of date .03 

17) Participate in annual review of subordinates' PDs .10 

21) PDs limit my flexibility in assigning work .14 

26) Know when subordinates' PDs aren't current .10 

30) Employees leaving for higher pay is a problem .54 

37) Complete set of PDs available to me .15 

39) Difference in pay over subordinates' is adequate .04 

46) Pay is enough to attract qualified people -.15 

49) My PD describes what I do -.08 
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Item 

No . Weight 

6. POSITION MANAGEMENT 

3) Certain functions should be combined .32 

28) Received training in PM -.13 

43) Some positions in my unit should do higher priority work .14 

45) Aware of PM objectives .02 

50) Could reorganize my unit to be more effective/efficient .10 

51) Enough authority to reassign ray employees .41 

56) Getting maximum utilization of employees .15 

57) My skills and abilities are well used -.04 

7. JOB INFORMATION AND PERFORMANCE EVALUATION 

10) Have soma unsatisfactory employees .01 

12) Periodically discuss performance with subordinates .39 

41) I see that subordinates know job requirements 

62) Boss lets me know when I do a good job -.01 

64) Know what is expected of me -.05 

8. COMMUNICATION 

5) Given "why" on info to me, to answer employee questions -.26 

22) My opinions are considered by management .20 

24) Usually get info from grapevine first .28 

33) Regularly attend supervisory staff meetings -.09 

38) Rules and regulations available in writing .10 

42) Have opportunity to help plan personnel policy -.15 

47) Get most info at the same time as employees .26 

52) Discuss changes with employees in advance .02 

9. SUPERVISION 

4) Used incentive awards system in past year .11 

8) Not aware of any complaints or dissatisfactions in my group 

not properly dealt with .19 

14) Have delegated authority appropriately -.04 

15) Personnel people have more say about ray employees than I do .04 

20) Easier to transfer unsatisfactory employee than to discipline 
or fire 
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.14 



Item 

No. 



Weight 



23) Workload leaves little time to help subordinates .09 

2 7) Prefer not being a supervisor, but only way to get higher 

grade . 30 

29) Employees free to bring grievances or appeals to me .21 

34) Enough disciplinary authority .22 

53) Trained in how to be a supervisor .15 

60) Enough backing/authority to do my job .07 

10. EMPLOYEE SERVICES 

36) Recreation facilities OK .16 

55) Medical/health facilities OK .24 

58) Eating facilities OK -.09 

61) Transportation facilities OK .36 

63) Parking facilities OK .31 

11. MORALE 

59) Morale in ray unit is high .32 

65) Would recommend this to friends as a place to work .32 
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TABLE 4 



SHORT INTERPRETATIONS OF THE ELEVEN FACTORS 

1. Merit Promotion: 

High score denotes dissatisfaction. 

2. Training: 

Confounded with authority but low score suggests satisfaction in 
program. 

3. Labor-Management Relations: 

Low score associated with satisfactory information about roles 
and treatment. 

4. Equal Employment Opportunity: 

High score associated with the perception that minority members 
do not perform as well as others. 

5. Classification and Pay: 

Load heavily on the question of employees leaving for higher pay 
elsewhere. High score indicates disagreement with that statement. 

6. Position Management: 

Driven largely by authority to reassign employees and combine 
functions. It is confounding, but basically a low score denotes 
satisfaction with this managerial environment. 

7. Job Information and Performance Evaluation: 

Loads on communication with subordinates. Low score indicates 
satisfactory communication is perceived by the supervisors. 

8. Communication: 

Refers to communication from above. Low score denotes poor 
communication. 

9. Supervision: 

Low score expresses satisfaction with their supervisory situation. 

10. Facilities: 

Transportation and parking seem to be the variable issues. Low 
score denotes satisfaction. 

11. Morale: 

Low score denotes high morale. 
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TABLE 5 



DEMOGRAPHIC DISTRIBUTIONS FOR THE SIX CLUSTERS 

Clusters 

1 2 3 4 5 6 



NUMBER OF PEOPLE IN CLUSTER 
Milit ary 

Civilian (default) 

SEX 

Male 

Female 

YEARS FEDERAL SERVICE 
0-3 
4-10 
11-20 
> 20 

EDUCATION 
High School 
College 
Postgraduate 

GENERAL WORK AREA 

Scientific and Eng Professional 

Other Professional 

Administrative 

Technician 

Clerical 

Trade 

GRADE RANGE 
GS 3-5 
GS 6-9 
GS 10-13 
GS 14 
Leader 
Foreman 

General Foreman 
Superintendent 

WHEN WERE YOU LAST PROMOTED 
Never 
0-5 

> 5 

WHEN DID YOU LAST RECEIVE TRAINING 
Never 
0-2 
3-5 

> 5 

MEMBER OF A MINORITY GROUP 
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TABLE 6 



OFFICE OF CIVILIAN MANPOWER MANAGEMENT SURVEY 



SUPERVISORS 



Vs. Navy Activity % 

Average 1 Yes No 



Navy % 
(11/17/71) 
Yes No 



1. Merit Promotion + 3% 

Satisfied with people referred +1 

Good applicants to choose from +3_ 

Candidates in reasonable time +1 

Trained in supervisory appraisals +7 

2. Training 4- 2% 

Job related and pays off +4 

Can get for employees in reasonable time +6 

Difficult to spare employees for off-job 

training -5 

3 . Labor-Management Relations +12% 

Personnel office assists in dealing with 

unions +13 

Kept informed of provisions of agreements TT3" 

Feel free to treat union members/non same +13 

Trained in Federal program +6 

Satisfactory dealings with unions +17 

4. EEO + 6% 

Supported by top management and supv f s +1 

Men and women have same job opportunities +9" 

Minority members perform as well as others j^6 

Would mind working for a minority supv +12 

Minority members treated fairly +3 

5 . Classification and Pay + 3% 

Know procedure when PD/JD out of date +8 

Participate in annual review of subor- 
dinates 1 PDs/JDs +3 

PDs/JDs limit my flexibility in assign- 
ing work +1 

Know when subordinates ’ PDs/JDs not current +6 

Employees leaving for higher pay is a problem +5 
Complete set of PDs/JDs available to me +5 

Difference in pay over subordinates’ is 

adequate +2 

Pay is enough to attract qualified employees +5 
My PD/JD describes what I do +1 

6 . Position Management + 5% 

Certain functions should be combined +3 

Received training in PM +3 

Some positions in my unit should do 

higher priority work 0 

Aware of PM objectives +9 

Could reorganize my unit to be more 

effective/efficient +1 

Enough authority to place/reassign my 

employees +2 
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Compared to underscored response. 
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Navy 


% 




Vs. Navy 


Activity % 


(11/17/71) 


SUPERVISORS 


Average 1 


Yes 


No 


Yes 


No 


Getting maximum utilization of employees 


+10 


66 


18 


56 


26 


My skills and abilities well used 


+8 


75 


13 


67 


20 


Job Information, Performance Evaluation 4- 6% 












Have some unsatisfactory employees 


+7 


23 


69 


30 


62 


Periodically discuss performance with 












subordinates 


+5 


84 


10 


79 


13 


I see that subordinates know job 












requirements 


+4 


94 


1 


90 


1 


Boss lets me know when I do good job 


+7 


59 


25 


52 


31 


Know what is expected of me 


+5 


89 


3 


84 


6 


Communication 4- 5% 












Given "why" on info to me, to answer 












employee questions 


+5 


70 


21 


65 


25 


My opinions are considered by management 


+6 


73 


12 


67 


18 


Usually get info from grapevine first 


+3 


34 


56 


38 


53 


Regularly attend supervisory staff meetings 


+5 


60 


34 


55 


37 


Rules and regulations available in writing 


+1 


77 


13 


76 


12 


Have opportunity to help plan personnel 












policy 


+7 


41 


43 


34 


49 


Get most info at same time as employees 


+11 


35 


57 


39 


46 


Discuss changes with employees in advance 


+3 


91 
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88 
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Supervision 4- 4% 












Used incentive awards system in past year 


+2 


60 


34 


58 


36 


Not aware of any grievances or appeals in 












my group not dealt with 


-3 


68 


24 


71 


20 


Have delegated authority appropriately 


+5 


94 
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89 
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Personnel people have more say about my 












employees than I do 


+3 


34 


54 


36 


51 


Easier to transfer unsatisfactory employee 












than discipline or fire him 


+8 


41 


36 
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28 


Workload leaves little time to help 












subordinates 


_0 


36 


58 


35 


58 


Prefer not being a supervisor, but only 












way to get higher grade 


_0 


17 
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16 


75 


Employees free to bring grievances or appeals 










to me 


-1 


95 
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94 


1 


Enough disciplinary authority 


+9 


73 


16 


64 


23 


Trained to be a supervisor 


+9 


84 
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75 


13 


Enough backing/authority to do my job 


+9 


76 


12 


67 


19 


Employee Services 4- 4% 












Recreation facilities OK 


+1 


45 


28 


44 


29 


Medical/health facilities OK 


+3 


75 


11 


72 


11 


Eating facilities OK 


+6 


50 


37 


44 


40 


Transportation facilities OK 


+4 


62 


22 


58 


24 


Parking facilities OK 


+5 


21 


19 


65 


24 


Morale + 5% 












Morale in my unit is high 


+7 


64 


18 


57 


25 


Would recommend this place to my friends 


+2 


70 


13 


68 


16 



31 



DISTRIBUTION LIST 



No. 



Headquarters, Naval Material Command (CP-5) 
NMAT-09 

Crystal Plaza #5 

2211 Jefferson Davis Highway 

Arlington, VA 20360 

Naval Supply Systems Command 
SUP(09T) 

Crystal Mall 3 

1931 Jefferson Davis Highway 
Arlington, VA 20376 

Assistant Comptroller 

Financial Management Systems 
Career Development 
Dr. Rodman (NCFT-3) 

Crystal Mall 3 

1931 Jefferson Davis Highway 
Arlington, VA 20367 

Office of Civilian Manpower Management (PLA-1) 

Deputy Director Capt, Richard Fay 

Pomponio Plaza 

1735 N. Lynn Street 

Arlington, VA 22209 

Naval Facilities Engineering Command Headquarters 
MARAVICH, Millicent NFAC 
11533 HOF Bg 2 
325-8573 

Hoffman Bldg. H 2 
200 Stovall Street 
Alexandria, VA 22332 

Office of Naval Research (BCT //l) 

Organizational Effectiveness Research Program 
0NR-452 

Ballston Center Tower //I 
800 N. Quincy Street 
Arlington, VA 22304 

Office of Assistant Secretary of Defense 
(Manpower and Reserve Affairs) 

Director of Manpower Programs 
Mr. I. M. Greenberg 
Pentagon 

Washington, D.C. 20350 



of Copies 
1 

1 

1 



1 

1 



1 



32 



No, of Copies 



Mrs, Carl Clewlow 

Deputy Assistant Secretary 

(Civilian Personnel Policy) 

3D281 Pentagon 
Washington, D.C, 20350 

Dr , Gundar King 
Dean, School of Business 
Pacific Lutheran University 
Tacoma, Washington 98447 

Naval Personnel Research and 
Development Center 
M. Wiskoff 
San Diego, CA 92152 

Naval Audit Service HQ 
Mr. J, Huff 

Falls Church, VA 20315 

Naval Supply Systems Command 
Library 

Crystal Mall 3 

1931 Jefferson Davis Highway 
Arlington, VA 20376 

Naval Facilities Engineering Command 

Library 12507 

Hoffman BLD #2 

200 Stoval Street 

Alexandria, VA 22332 

Bureau of Naval Personnel 

Library 

1705 AA 

Arlington Annex 
Arlington, VA 20370 

Defense Documentation Center 
Cameron Station 
Alexandria, VA 22314 

Chief of Naval Research 
Department of the Navy 
Arlington, VA 22217 

Library (Code 0212) 

Naval Postgraduate School 
Monterey, CA 93940 

33 



1 

1 

1 

1 

1 

1 

1 

2 

2 

2 



No > of Copies 

Dean of Research 1 

Code 023 

Naval Postgraduate School 
Monterey, CA 93940 

Library (Code 55) 2 

Department of Operations Research 
and Administrative Sciences 
Naval Postgraduate School 
Monterey, CA 93940 

Professor D. P. Gaver, 1 

Professor M. S. Wortman, 1 

Professor Robert R. Read, 1 

Professor William H. Githens, 1 

Professor Richard S. Elster, 1 

Professor Gerald L. Musgrave and 5 

Professor John W. Creighton 1 

Department of Operations Research 
and Administrative Sciences 
Naval Postgraduate School 
Monterey, CA 93940 



34 



4 aprco 



2 6 2 7 ? 



U188136 



Naval Postgraduate School. 
NPS-55Re73061. 

4 APR 80 2 6 2 7 



l) 1 88136 



DUDLEY KNOX LIBRARY - RESEARCH REPORTS 



5 6853 0106241 





